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a region of interest of the skin, based on direct digital 
imaging of the region of interest or the digitization of color 
photographic slides of the region of interest, illuminated by 
appropriately filtered light. Preferably, a digital image at a 
low spectral band is automatically segmented and that 
segmented mask is used to segment the other images by a 
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automatically estimated. The region of interest is automati- 
cally characterized by the digital processor, based on those 
parameters. The region of interest may include a skin lesion, 
in which case the present inyention enables the character- 
ization of the lesion as malignant or benign. 
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SYSTEMS AND METHODS FOR THE 
MULTISPECTRAL IMAGING AND 
CHARACTERIZATION OF SKIN TISSUE 

This application claims the benefit of U.S. provisional 5 
application Ser. Nos.' 60/039,2 1 ! 8 and 60/039,407, both of 
which were filed on Feb. 28, 1997, and are incorporated by 
reference, herein. 

STATEMENT REGARDING FEDERALLY to 
SPONSORED RESEARCH 

This invention was made with Government support under 
NIH/National Cancer Institute Contract No. 2-R44- 
CA60229-02A1 and U.S.A.F. Phillips Laboratory Contract 
No. F29601-95-C-0125. The Government has certain rights 
in this invention. 

FIELD OF THE INVENTION 

This invention relates to methods and systems for the 20 
computer controlled image analysis of digital skin tissue at 
a plurality of wavelengths, which may include those outside 
of the red-green-blue bands. The methods and systems 
further include the automatic characterization of the condi- 
tion of the skin tissue, based on automatically computed 25 
values of parameters which are functions of characteristics 
of the skin tissue, based on the digital images. Skin lesions 
can be analyzed for determining whether the lesion is a 
melanoma, for example. Systems for digitally imaging and 
analyzing skin tissue are disclosed, as well. 30 

BACKGROUND OF THE INVENTION 

Melanoma is a usually fatal skin cancer, unless it is 
detected and surgically removed in its earliest stages. Early 
detection of malignant melanoma is difficult because early 35 
melanomas, those having a Breslow thickness less than 1 
mm, share many diagnostic features with benign lesions, 
such as dysplastic nevi or atypical melanocyte nevi. 

To aid in the analysis of lesions, conventional 4Q 
photography, referred to as "clinical imaging", has been 
used to image the lesion for further study. The effectiveness 
of clinical imaging can be compromised, however, by specu- 
lar reflection by the skin. Polarizers have been used for 
polarized imaging, which minimizes specular reflection. 45 
. Dermoscopy is another technique for examining skin, in 
which specular reflection is minimized. Dermoscopy also 
assists in clinically differentiating melanoma from its benign 
simulants by enabling the observation of features of pig- 
mented melanocytic lesions that are not discernible by the 50 
naked eye. In dermoscopy, the skin is made more transparent 
to light by providing an oil layer over the skin, in front of the 
optical system. A glass plate is placed over the oil layer. The 
oil has an index of refraction between the index of refraction 
of the horny layer of the skin and the glass plate. Standard 55 
magnifying optics may be used to enlarge the structures 
rendered visible on and under the surface of the skin by the 
oil layer. The region of interest can then be examined 
visually. Slides of the region of interest can be made, as well, 
for future study. 60 

Despite their similarities, most malignant melanomas 
differ in certain of their characteristics from other melano- 
cytic lesions. A major advance in characterizing skin lesions 
based on certain of the observable differences between 
malignant and other lesions is the "ABCD" rule, where 65 
A=asymmetry, B=border irregularity, C=color variability, 
and D=diameter greater than 6 mm. A corresponding ABCD 
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rule, where "D" refers to dermoscopic structures, such as 
brown globules, black dots or pigment networks within the 
lesion, is applied to dermoscopic images. Because the clini- 
cal and dermoscopic applications of these rules are 
subjective, they are not very reliable. 

When skin is illuminated by light, the light can be 
re -emitted by reflection, scattering or fluorescence. It is 
known in the art that the re-emission of light absorbed at 
different wavelengths by a region of interest of skin can 
provide different information. For example, as the wave- 
length of the light increases, its depth of penetration into the 
skin or other tissue also increases. Chromophores at different 
depths in the tissue therefore absorb and re-emit light at 
various wavelengths. Melanin and hemoglobin are examples 
of such chromophores. 

Since the unaided eye cannot perceive light outside of the 
visible region or low-contrast structure in visible-light 
images, information which may be useful in diagnosing a 
lesion may not be directly observable. Digital acquisition 
and processing of dermoscopic images may, therefore, 
improve diagnostic reliability by employing more of the 
information residing in such images that is not directly 
observable. There have therefore been attempts to use 
objective, computer-based, image analysis algorithms that 
can discern meaningful differences between benign and 
malignant melanocytic lesions with sufficient accuracy. 

Computer processing of images requires that the image be 
in digital form. A digital image is an array of digital signals 
whose values are a function of certain characteristics of the 
subject of the image. When imaging skin lesions, the digital 
images comprise digital signals whose values are a function 
of the re-emission characteristics of the skin and lesion, at 
different spectral bands of light. The array is obtained by 
spatial sampling and quantizing the intensity of images 
obtained with film or directly by electronic cameras. Prac- 
tical limitations on the number of picture elements or pixels 
per unit area of image determine the achievable spatial 
resolution of the digital image. The digital image typically 
needs to be segmented to separate the digital signals which 
are a function of the skin lesion from the digital signals 
which are a function of the surrounding skin. 

Computer aided analysis has also been used to classify 
skin lesions using quantitative values indicative of particular 
characteristics of lesions, referred to as parameters. Based 
on histopathological diagnosis of lesions, algorithms have 
been developed which use linear or non-linear classifiers to 
combine parameters provided by the operator of an imaging 
device or a physician or computed by a processor, to yield 
a value which can be used to classify the lesion. Because 
some of the steps in the computer-aided analysis of which 
we are aware depend on subjective judgments of an 
individual, such analyses may provide highly variable 
results. 

The images heretofore available have been obtained with 
commercially available red-green-blue color imaging appa- 
ratus. Color photographic transparencies of skin lesions 
have been digitized and skin lesions have been directly 
imaged with "three-chip" digitizing cameras. Such cameras 
employ broad-band filter bandpasses that are ultimately 
based on the wavelength response of the human visual 
system and have large regions of overlap. 

Electronic images may also be obtained in narrower, 
non-overlapping filter bandpasses, which may reveal 
additional, wavelength-dependent differences between the 
images of melanomas and of benign lesions. However, such 
devices have had poor resolution and/or poor signal-to-noise 
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characteristics which prevent the acquisition of digital 
images of melanocytic skin lesions of sufficient quality for 
effective application of machine vision techniques for lesion 
diagnosis. 

Existing imaging systems and processes also tend to 5 
suffer from an inability to provide the required repeatability 
of the values of extracted lesion parameters, due in part to 
a lack of standardization with respect to spatially varying 
artifacts. The parameters, therefore, lack in variance to light- 
ing and image exposure conditions, for example. Obtaining 10 
high signal-to-noise ratios in images recorded in narrow 
filter bandpasses, when exposure times are sufficiently short 
that the skin is effectively "frozen" during the exposure 
sequence, has also been difficult. In addition, since the 
optimum wavelengths for automatic characterization may 15 
not be the optimum wavelengths for visual observation, it 
may be difficult to reconstruct high-fidelity color images 
from the digital images for visual interpretation by a clini- 
cian. 

The assessment of wounds and burns through the appear- 
ance of color images present similar challenges. Existing 
technology for the imaging of skin in vivo for these purposes 
is also inadequate. Practical solutions to the problems of 
employing multispectral digital imaging of skin for the 
analysis of lesions, wounds, or other conditions have not " 5 
been found. 

SUMMARY OF THE INVENTION 

The methods and systems of the present invention provide 30 
for the acquisition of digital images of skin at a plurality of 
spectral bands to automatically characterize the condition of 
the tissue based on the digital images. Spectral wavelength 
bands within and outside of the visible band may be used. In 
accordance with the present invention, a pigmented skin 35 
lesion can be characterized as malignant or benign, for 
example. The digital images comprise a plurality of digital 
signals whose values are functions of the condition of the 
tissue. The digital images acquired are subjected to objective 
and quantitative analysis by a digital processor to detect and 40 
identify abnormalities. The analysis includes image 
segmentation, parameter estimation and characterization of 
the skin. The estimation and characterization steps are 
automatic. The segmentation step may be automatic, as well. 
Subjective judgments are therefore minimized or eliminated. 45 

It has been found that generating the segmentation mask 
from a digital image acquired with light in a spectral band 
which does not penetrate deeply into the skin, such as a blue 
spectral band, provides superior results. After segmentation, 
estimated values which are functions of characteristics of the 50 
lesion, such as its texture, asymmetry, blotchiness, and 
border irregularities, are computed and used to automati- 
cally characterize the condition of the skin. Digital signals 
corresponding to hair or blob-like structures are preferably 
removed during segmentation. 55 

In accordance with the present invention, a method for 
characterizing the condition of a region of interest of the 
skin, wherein the absorption and scattering of light in 
different spectral bands by the region of interest is a function 
of the condition of the skin, is disclosed. The method 60 
comprises illuminating the region of interest of the skin by 
light in at least three spectral bands and digitally imaging the 
region of interest at the at least three spectral bands with the 
light re-emitted by the skin to generate digital images 
comprising digital signals whose values are a function of the 65 
condition of the skin. The digital images are provided to a 
processor which segments the digital images by generating 
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a segmentation mask from a digital image in any one of the 
at least three spectral bands, computes at least one estimated 
value for each digital image at each spectral band which is 
a function of a characteristic of the region of interest within 
the segmentation mask, characterizes the condition of the 
skin . based on the estimated values, and outputs the charac- 
terization of the condition of the skin. Preferably, the 
segmenting, estimating and characterizing steps are con- 
ducted without the intervention of an operator. Useful 
parameters include measures of the texture, asymmetry, 
blotchiness and border irregularity of the portion of the 
region of interest. 

The digital images may be obtained by directly imaging 
the region of interest with a digital camera, or digitally 
imaging color slides of the region of interest, through 
appropriately filtered light. 

The characterizing step may include comparing a 
weighted combination of the parameter values against a 
threshold value. The weight coefficients for each parameter 
value and the threshold value may be selected based on a 
training set of images of lesions or other skin conditions, 
whose condition has been determined, preferably through 
histological examination by a plurality of doctors. 
Preferably, specificity is maximized under the constraint of 
100% sensitivity to melanoma. 

In accordance with another aspect of the invention, a 
system for characterizing the condition of a region of 
interest of skin includes means for illuminating the region of 
interest with light in at least three spectral bands and a 
camera for acquiring digital images of the region of interest 
based on the light re-emitted from the illuminated region of 
interest at each of the spectral bands. The digital image 
comprises digital signals whose values are a function of the 
condition of the region of interest. A digital processor 
segments the digital images by generating a segmentation 
mask from a digital image in any one of the at least three 
spectral bands, and computes at least one estimated value for 
each digital image at each spectral band which is a function 
of the texture of the portion of the region of interest within 
the segmentation mask. The processor characterizes the 
lesion based on the estimated value or values. The other 
parameters discussed above may be used, as well. 

The camera may be a single-chip or multiple-chip charge- 
coupled device which detects light in a plurality of spectral 
bands between the near ultraviolet to near infrared. The filter 
means may be a plurality of interference filters mounted on 
a wheel for stepping any filter into a position intercepting the 
light from the light source. Preferably, at least one of the 
spectral bands has a center which lies between about 400 and 
500 nanometers, at least one of the spectral bands has a 
center which lies between about 500 and 600 nanometers, 
and at least one other spectral band has a center which lies 
between about 750 and 1000 nanometers. 

DESCRIPTION OF THE FIGURES 

FIG. 1(a) is a schematic illustration of a method and 
system of imaging a region of interest of skin in accordance 
with the present invention; 

FIG. 1(b) is a schematic illustration of a plurality of 
narrow spectral bandwidths which may be used to illuminate 
the skin in the embodiment of FIG. 1(a); 

FIG. 1(c) is a schematic illustration of alternative methods 
and systems for digitizing and analyzing color photographic 
slides of a region of interest of skin; 

FIG. 2 is a schematic illustration of preferred illumination 
and imaging portions of a computer controlled imaging 
system for direct imaging of a lesion; 
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FIG. 3(a) is a flow chart of a calibration procedure for use M, that are preferably effectively non-overlapping, as shown 

with the present invention; schematically in FIG. 1(b). The skin is illuminated by a 

FIG. 3(b) is a flow chart of a method of processing images f ^ 5* * hi !? H S ht * h . ich « b X narT0W passband 

for classifying lesions as malignant or benign, in accordance s 4 The filtered light is preferably conveyed to the skin 

with the present invention; 5 2 : through a fiberoptic ihuminator 5. The light re-emitted by 

r the illuminated skin is imaged by a low-noise, high- 

FIGS. 4(a) and 4(b) are histograms of a malignant mela- resolution monochrome camera 6, which is preferably an 

noma and of an atypical melanocyte nevus, respectively, electronic charge-coupled ("CCD") camera. Digital images 

showing two peaks in each histogram; output by the camera 6 are provided to a computer 12 for 

FIGS. 5(a) and 5(b) are histograms of another malignant 1Q processing, 

melanoma and another atypical melanocytic nevus, The computer 12 includes a digital interface 12a, a 

respectively, showing three or more peaks in each histo- memory 12b and a digital processor 12c. A display 19 is 

gram; preferably provided as well. The computer 12 includes an 

FIGS. 6(a) and 6(d) are digital images in the blue spectral m P ut t0 A a di S itaI ^efface for receiving the digital 

band of another malignant melanoma and another atypical 15 im P a S es * A mem ^ 12b stores ^ digital images, and the 

melanocytic nevus, respectively; software controlling operation of *e imaging system the 

J r j image processing, and the classification and characterization 

FIGS. 6(b) and 6(e) are digital images of the images of of me lesion ^ digital processor 12c, under control of the 

FIGS. 6(a) and 6(d) respectively, after thresholding; software, performs the calculations. The computer 12 has an 

FIGS. 6(c) and 6(f) are digital images of the images of output connected to a display 19, which can display the 

FIGS. 6(a) and 6(d), respectively, after iterative threshold- 20 processed images and the results of the classification and 

ing; characterization procedures for each image. The computer 

FIGS. 7(a) and 1(d) are digital images in the blue spectral "also preferably has outputs connected to the source of 

band of another malignant melanoma and another atypical ^ 3 T 1 the cam fr em ' °T ° g ^ kTk?™ 

, . * Jr level and exposure times, respectively, as described below, 

melanocytic nevus; ■ . r . , r / . . 

, . 25 The image processing, classification or characterization 
FIGS. 1(b) and 7(c) are digital images of FIGS. 7(a) and and other programs can be implemented on a personal 
1(d), respectively, resulting from iterative processing and computer, using a programming language, such as FOR- 
showing dark blobs outside the lesion area; TRAN or C. The memory 12b which stores the software can 
FIGS. 7(c) and 1(f) are digital image masks of FIGS. 1(b) be any convenient media readable by the computer 12, such 
and 1(d), respectively, resulting from image cleaning; 30 as the hard drive of the computer, read only memory, random 
FIGS. 8(a) and 8(c) are digital images in the blue spectral access memory with a battery backup, electrically pro- 
band of another malignant melanoma and another atypical grammed ROM, electrically erasable ROM, floppy disc, or 
melanocytic nevus, respectively, showing hair; CD ROM. Other suitable media may be used, as well. 

FIGS. S(b) and 8(f) are reverse intensity contrast images W^ 11 * e fiIter bandpasses have minimal overlap, as in 
of the lesions of FIGS. 8(a) and 8(c), respectively; 35 FIG. 1(b), each monochromatic image will contain spec- 
tre as \ a o/ ^ a i • w f trally independent information. Such spectral separation is 
FIGS. 8(c) and 8(2) are digital images resulting from an , , *V * , r j «- • i j- f , . , . 

i • j „ r Tzir- c o/ \ j believed to be useful for differential diagnosis of skin lesions 

averaging process applied to the images of FIGS. 8(a) and * u * * • * * i • ^ *u i u- 

q,l\ f u • that contain varying amounts of melanin, and of hemoglobin 

8(o), to remove hair; . ■ r . ° A ^ c . „ . . 

m different oxidation states, for example. Spectral separation 

FIGS. S(d) and S(h) are binary lesion masks resulting 4Q is ^ ^beved to be useful in distinguishing granulation of 

from the segmentation of the images of FIGS. 8(c) and 8(g), tissue md other strU crural details of wounds in various 

respectively; stages of healing. One or more of the wavelength bands may 

FIG. 9 is a spatial filter used to remove hair; lie outside the visible region, such as in the near infrared 

FIGS. 10(a)-10(c) are segmented digital images in the and/or the near ultraviolet, as long as the wavelength is 

blue, green and red spectral bands, of the malignant mela- 45 within the response range of the combined optical system 

noma whose histogram is shown in FIG. 5(a); including the electronic camera 6. 

FIGS. 10(a>-10(/) are segmented digital images in the In accordance with another aspect of the invention, the 

blue, green and red spectral bands, of an atypical melano- digital images of skin lesions can be derived from color 

cytic nevus whose histogram is shown in FIG. 5(b); slides of the lesions obtained by clinical imaging, 

FIG. 11 is a chart of lesion parameters and their associated 50 dermoscopy, or polarization imaging. FIG. 1(c) is a sche- 

diagnostic accuracy, sensitivity and specificity when used matic lUustratl ° n of alternative approaches to the acquisition 

individually a digitization of images of skin lesions from color slides. 

, ' . , , A photo camera 13 produces 35-mm color slides of a region 

FIG. 12 is a plot of linear classifier values versus lesion of me skin 14 ^ camera 13 can be a D hot® camer a 

identification number, for 41 malignant melanomas and 104 55 from Hdne 0 totechnik Gmbh & Ca AG , Ge for 

atypical melanocytic nevi; example ^ ^ m typica]ly stQred in m J5 

FIG. 13 is a plot of linear classifier values versus lesion ^ slides ^ subsequently reimaged by a monochrome 

identification number for 24 superficial spreading melano- camera 16> which may 5e a CCD cameraT mat photographs 

mas and 16 melanomas ui-situ; and each slide as it is Ruminated by white light that has passed 

FIG. 14 is a plot of lesion parameter versus Breslow 60 through a sequence of bandpass filters 17 to create a color 

thickness for 24 superficial spreading melanomas. . filtered version of the image. The slides can be illuminated 

DESCRIPTION OF THE INVENTION * br ° ad ° r naiT0W blUC (B) ' **** (G) red (R) WaVC " 

uti^CKiKi iuin vr i UN vnw i iuin kngth respectively. The broad wavelength bands may 

FIG. 1(a) is a schematic illustration of a method and overlap somewhat. In one example, the blue wavelength 

system 1 in accordance with the present invention, by which 65 band was about 400 nm±30 nm, the green wavelength band 

images of the skin 2 are acquired by a camera nearly was about 550 nm±30 nm, and the red wavelength band was 

simultaneously at a plurality of spectral bands, X t , i=l,2, . . . about 700 nm±30 nm. 
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Each of the filtered representations is recorded by the 
monochrome camera 16, which provides the resulting digital 
images 18 to an input of the computer 12. If a CCD camera 
is not used, the slide images could be digitized by any 
available commercial digitizer including three channels, one 5 
for red, one for green and one for blue, as long as the pixel 
size in the lesion plane after digitization is less than about 60 
micrometers ("urn")- 

An appropriate CCD camera 16 is available from 
Electrim, Inc., Princeton, NJ. The camera 16 has a photo- 10 
graphic macro-lens, wherein f#/2.8 and f=100 mm. 
Preferably, the spatial resolution of the CCD camera 16 
provides pixels having a size about 10-30 um in the lesion 
plane. The CCD camera 16 from Electrim, Inc., has 753x488 
pixels. The spatial resolution with such a camera is approxi- 15 
mately 21x24 um at the lesion plane. Digital images of 
lesions obtained with this imaging system were used to 
classify lesions as malignant or benign, and to characterize 
lesions as invasive or non-invasive, as described further, 
below. The Electrim, Inc., CCD camera has 16 has rectan- 20 
gular pixels. A CCD camera with square pixels would 
simplify the calculating procedures. 

Alternatively, a 3-chip CCD camera 20, indicated in 
phantom in FIG. lc, may be used to reimage the slides of the 
region of interest. The CCD camera 20 provides digitized 
images for subsequent analysis by the computer 12. Broad 
bandpass filters, which are part of the CCD camera 20, 
produce a representation of the lesion as a set of three 
narrowband images. The filters are typically in accordance 
with CIE Standard Observer, wherein the bandwidths are 30 
broad. 

FIG. 2 is a schematic illustration of the illumination and 
imaging portions of a preferred computer controlled imaging 
system 22 in accordance with the present invention, for ^ 
imaging a region of interest of skin including a lesion. The 
electronic camera 23 may be a 10-bit monochromatic elec- 
tronic CCD camera 23, such as the Xillix Model 1400, 
available from Xillix Technologies Corp., Canada. The 
Xillix camera is equipped with wide band, low distortion 4Q 
foreoptics, such as the XJP 1.9/0501, available from Jos. 
. Schneider Werke, Germany. The lower distortion fore optics 
and the camera minimize chromatic aberrations of the 
optical system over the "multispectrar sequence of 
exposures, enabling registration of images with sub-pixel 
accuracy, over the entire field of view. 

To ensure repeatability of imaging conditions and to 
minimize required intervention by the operator, it is pre- 
ferred that the system be operated at a preset f/stop. For 
cameras such as the Xillix Model 1400, exposure times are 50 
preferably controlled by the computer 12 through an elec- 
tromechanical shutter that can operate reliably between 
minimum and maximum exposure times t ntin and t m ^ v . 

The imaging system provides low-noise, high-resolution 
digital images at high data transfer rates, with low distortion 55 
imaging over the entire range of wavelengths covered by the 
collection of filters. The Xillix camera, discussed above, has 
a resolution at the skin surface of about 20 microns per pixel. 

The CCD camera 23 is preferably contained in a hand- 
held unit, represented schematically as box 24. The illumi- 60 
nator source 25 is a tungsten-halogen lamp whose intensity 
is controlled by a light-stabilized power supply 26 whose 
setting is automatically adjusted by the computer 12. A 150 
watt lamp, such as the Phillips EJA, available from Phillips 
Electronics North America Corporation, N.Y., may be used, 65 
for example. The output of the lamp 25 is white light A 
narrowband filter 27 is provided between the source and an 
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optical fiber 28. A plurality of narrowband filters, each one 
corresponding to a different spectral wavelength band, are 
mounted on a filter wheel 29. Preferred filter bandwidths are 
listed in Table 1, below. The filter wheel 29, which is driven 
by a stepping motor 29a, advances each filter to its proper 
position between the lamp 25 and the optical fiber 28, and 
holds each filter in position for a sufficient period of time. 
The computer 12 controls the motor 29a. More or fewer 
filters may be used. Appropriate lenses 14 are provided 
between the lamp 25 and the filter 27, and between the filter 
27 and the optical fibers 28, as well. One or more fiber 
illuminators are provided for conveying the light from the 
source to the lesion. Two such illuminators 30a 30/? are 
shown in FIG. 2 for simplicity. Although the fiber illumi- 
nator illustrated is a bifurcated pair, a ring illuminator which 
provides more nearly uniform illumination at the skin 
surface, is preferred. An angle of illumination of about 20° 
is also preferred. A Fostec Model A0603 available from 
Fostec, Inc., N.Y., may be used, for example. 

The hand-held portion of the system 24 of FIG. 2, which 
includes the camera 23, may be mounted on a cantilevered 
arm (not shown) that can be locked into position. 

The digital signals making up each of the digital images 
output from the camera 23 are provided to the computer 12. 
The computer 12 conducts image processing procedures on 
the digital images to calibrate the images, and to objectively 
segment, estimate parameters, and classify the lesions based 
on the estimated parameters. Operator judgment is not 
required at any point in the process. 

Control is maintained by the computer 12 over source 
intensity, filter position, and such camera settings as shutter 
timing, through the digital interface 12a. Key control param- 
eters are empirically chosen on the basis of feedback from 
histograms of trial images. The intensity of the lamp 25 may 
be maintained at a stable value, commensurate with the 
10-bit dynamic range of the camera 26, by monitoring a 
secondary light source, connected electrically in series with 
the primary light source 25. The light output from the 
secondary source is monitored by a light sensor that is 
optically isolated from light reflections associated with the 
primary source. Such reflections may be caused by the filters 
that are located on the filter wheel, or from the housing of 
the primary light source. This method provides optical 
feedback which is sensitive to changes in light intensity 
caused by changes in lamp lead resistance, for example, 
while it is insensitive to the variable amounts of fight 
reflected from the filters, for example. By means of a closed 
control loop, the optical feedback from the secondary source 
is used to maintain long-term constant light output from the 
primary source. 

The apparatus of FIG, 2 can be used for either clinical 
imaging of the skin, wherein the skin is imaged directly, 
dermoscopic imaging, wherein a layer of oil is provided over 
the skin and a layer of glass placed over the oil layer, or 
polarized imaging, where a polarizer 31 is added to mini- 
mize specular reflection as shown in FIG. 2. In dermoscopic 
imaging, the index-matching oil sufficiently reduces the 
specular reflection to avoid the need for a polarizer. 

Instead of being positioned between the light source 25 
and the optical fiber 28, the narrow bandpass filters 27 may 
be placed between the skin and the CCD camera 23 to filter 
the light reflected, scattered and radiated from the skin 2. 

The front end of the system preferably consists of a flat 
glass plate (not shown) for being placed over the skin. Light 
pressure is applied through the glass, onto the skin, through- 
out the imaging process. This helps to stabilize the region of 
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interest against unwanted motion which could blur an image 
or which could lead to spatial misregistration between 
images obtained in different filter bandpasses. 

The preferred filters 27 for lesion imaging with a 
tungsten-halogen white light source 25 have the center 
wavelengths X,- and bandwidths (FWHM) listed in Table 1, 
for i=l,2, . . . , M, M=10, wherein the bands are labeled by 
j=i— 1=0, 1, . . . M-l. Such filters are available, for example, 
from Intor, Inc., Tucson, Ariz. In each band, the exposure 
time is preferably selected to avoid saturation of the detector 
elements of the CCD camera 23, as well as to maximize the 
linear dynamic range over which the image data are 
recorded. These exposure times should be constrained to be 
within limits t mift and x max which are related to the electro- 
mechanical design of the shutter, optical throughput of the 
camera 23, and avoidance of image blur associated with 
motion during the exposure sequence. Suitable values of t,„,-„ 
and i max could be 10 ms and 550 ms, respectively, for 
example. The choice of center wavelength and FWHM for 
the filter channels, as well as the corresponding exposure 
times, should preferably also take into account the following 
considerations: 

(a) The center wavelength and FWHM for at least two 
channels should be chosen so that characteristic absorp- 
tion lines can be differentiated, such as those associated 
with melanin and hemoglobin; 

(b) For a given set of center wavelengths, there are upper 
limits on the associated bandwidths if spectral inde- 
pendence of data in different channels is to be 
maintained, as illustrated in FIG. 1(a); 

(c) Bandpasses should be chosen in the red, green and 
blue portions of the spectrum which enable "true-color" 
reconstruction of skin images that are suitable for 
visualization by clinicians; 

(d) The need for high signal-to-noise ratio in each image 
sets practical lower limits on the product of exposure 
time and filter bandwidth, especially at short 
wavelengths, where detector response falls off and 
lesion reflectance is low; and 

(e) The total time taken to acquire the images in all filter 
bands is preferably less than about three minutes, to 
minimize patient discomfort and possible motion. 

Based on considerations (d) and (e) above, and also taking 
into account the varying spectral reflectances of skin of 45 
different colors, the exposure times in each filter channel are 
preferably adjustable, with settings based on the dynamic 
range achieved on an empirical basis, with trial images. In 
this manner, both dynamic range and signal-to-noise ratio 
can be maximized for each filter channel. The preferred 50 
method is to choose l expi by iteration, based on intensity 
histograms of images of the skin obtained with trial expo- 
sures at each wavelength band. The histograms are analyzed 
to determine the number of pixels at the saturation intensity 
level, \ saJ =2 b -\ (1023 for b=10 bits). The exposure time is 55 
decreased if the number of saturated pixels exceeds a 
predetermined amount, such as 0.01% of the total. 
Conversely, to maintain high signal-to-noise ratio, the expo- 
sure time is increased if a predetermined percentile in the 
histogram, 99.9%, for example, is reached at less than a 
preset threshold, such as 99.5% of l sar The iteration process 
typically converges after two or three trials. 

The preferred exposure times at each wavelength for 
imaging skin of different colors to classify melanomas are 
listed in Table 1, for the embodiment of FIG. 2 with 10 
filters. It has been found that for the blue channel centered 
at 450 nm, the optimal exposure time for dark skin is 273 ms, 
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which is more than double the optimal 107 ms exposure time 
for light skin. On the other hand; in the near infrared channel 
centered at 780 nm, the exposure times listed are much 
shorter, between 24 and 35 ms, and vary relatively little with 
skin type. The optimal exposure time for dark skin in the 
deep blue channel at 430 rim is at t^ x ^550'ms, : duetto' the 
low skin reflectance and relatively low optical throughput of 
the system at this short wavelength. Even with an exposure 
time this long, therefore, the image is less than fully 
exposed. Greater throughput at this wavelength could be 
achieved, at the expense of poorer response in the infrared. 

In Table 1, the FWHM at 450 nm, is 100 nm, which is 
much broader than for other wavelengths. It has been found 
that where images are desired for visual analysis as well as 
computer processing, the broad wavelength band at 450 nm 
more closely matches the blue response of the human eye 
and is therefore preferred. In addition, the broad wavelength 
band provides data at a higher signal-to-noise ratio. 

Table 1 appears below: 
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Filter 



Optimal Exposure Times (msec) vs. Skin Color 
Center Filter Very 



Number 


Wavelength 


FWHM 


Ught 


Medium 


Tan 


Dark 


(j = i-D 


(nm) 


(nm) 


Skin 


Skin 


Skin 


Skin 


0 


430 


60 


405.2 


436.5 


484.4 


550.0 


9 


450 


100 


106.8 


124.9 


' 156.1 


273.3 


1 


500 


40 


56.4 


62.9 


88.7 


130.7 


2 


550 


10 


44.2 


50.4 


71.3 


92.9 


3 


600 


10 


19.1 


24.0 


29.6 


39.2 


8 


650 


10 


74.5 


92.3 


104.9 


132.0 


4 


700 


10 


71.6 


86.0 


98.0 


114.4 


5 


780 


30 


25.8 


29.1 


34.9 


23.6 


6 


880 


50 


34.1 


38.6 


44.8 


46.1 


7 


950 


60 


161.1 


187.6 


205.8 


212.0 



Tables similar to Table 1 can be readily constructed based 
on experimental results for other applications, where other 
spectral bands may be better suited. For example, in the 
analysis of wound healing, the ability to distinguish oxy- 
genated from deoxygenated blood would be desirable. In 
addition, the wavelengths and exposure times in Table 1 
reflect a balance between the best results for subsequent 
analysis of the images by a computer, and the best results for 
visual observation of the images. If visual observation is not 
necessary, other wavelength bands and exposure times may 
l?e used. 

FIG. 3(a) describes how the systems and methods of the 
present invention provide for calibration of the recorded 
images. The calibration procedure permits 10-bit image data 
to be recorded over a large linear dynamic range in each 
spectral band, independent of skin type. The recorded 
images can also be standardized for diffuse spectral reflec- 
tance. Consistent measures of reflectance ratios in different 
spectral bands can therefore be obtained, despite variations 
in illumination pattern with wavelength, changes in the 
position of the illuminator, or aging of the lamp, for 
example. 

First, the effects of dark current and "fixed pattern noise" 
are removed in Step 1. N images are recorded by the camera 
without illumination. Preferably 8 such dark images are 
recorded. The average of these N dark images, ID is calcu- 
lated and stored in the computer 12. 

Second, spatial inhomogeneities in the illumination and in 
the response associated with each CCD pixel are removed in 
Step 2. A sequence of N' images of an illuminated flat, 
diffuse reflectance standard, such as a white Spectralon® 
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target (R>99%) recorded. As above, N' is preferably 8. The 
N' images are recorded at each wavelength band. To average 
over local inhomogeneities in the reflectance standard, the 
target is moved continuously during the integration time and 
between exposures. A small motor, such as a reciprocating 5 
motor, may" be used. The integration" time and/or lamp 
intensity are adjusted by the computer 12 at each wavelength 
band until negligibly few of the pixels are at or just below 
an intensity level corresponding to saturation. These N' 
"flat-field" images are averaged to reduce the effect of 10 
spatial non-uniformities in the reflectance standard, as well 
as to improve the detection signal-to-noise ratio. The result- 
ing averages are stored in the computer as where i=l, 
2 M. 

Next, monochromatic "raw data" images of the skin, l sji 15 
are captured by the camera and digitally acquired by the' 
computer 12 within each filter passband, i=l,2, ... M. If 
dermoscopic imaging is used, where a thin layer of mineral 

011 is spread between the skin and a cover glass is fixed in 
position in front of the camera, each image of the skin 20 
preferably contains an image of a narrow strip of oil-free, 
diffusely reflecting gray material, held in place on the inside 
surface of the cover glass, and located along one edge of the 
field of view. The material may be cut out of a Kodak "18% 
gray" card. Dermoscopic imaging is preferred for melano- 25 
cytic lesions. The alternative clinical imaging mode is 
preferred for the imaging of wounds and burns because 
contact with the wound or bum by a cover glass is not 
desired. Although FIG. 2 indicates a lesion present on the 
skin 2, it will be readily understood that the same method 30 
will apply when a wound or burn is present, instead. In the 
clinical imaging mode, it is preferable to reduce specular 
reflections by employing the polarizer 31, as indicated in 
FIG. 2. 

In either the dermoscopic or clinical imaging techniques, 35 
a fourth step is preferably provided, in which the raw data 
is compensated for dark current and fixed pattern noise and 
then normalized to the flat-field images. The dark-field 
compensation is performed by subtracting the stored average 
dark image 1^, both from the flat-field image l wi and from the 40 
raw data image l si . The ratio of the results of these subtrac- 
tions is then taken. This standardizes the dark-corrected raw 
data to the flat-field image, compensating for spatially 
varying illumination and pixel-to-pixel response variations. 
After the ratio is taken, the result is standardized to the 45 
maximum level, 2^-1 which equals 1023 where b=10 in a 
10-bit data representation. The normalization process thus 
converts the image of the skin and the gray strip into a 
standardized diffuse reflectance map, with the result pre- 
serving a large linear recording dynamic range.. In FIG. 3(a), 50 
the dark-field corrected and flat-field-normalized images, 
also referred to as "flat-field-calibrated" images, are denoted 
as I-,.. In any image, standardization to maximum level can 
be reinterpreted directly in terms of equivalent diffuse 
reflectance on the basis of the average gray level over the 55 
image of the gray strip, <Igray strip> / and the measured 
average diffuse reflectance of the gray strip, which is 
approximately 0.2 and varies in a known and repeatable 
manner with wavelength. 

Preferably, the average image intensity in the gray-strip 60 
region is also used to calculate weighting factors for com- 
bining three or more monochromatic images to provide 
"true-color" visualizations of lesion images on the computer 

12 and display 19. This is preferably accomplished in Step 

5, where the user selects the spectral bands to be used in the 65 
color visualization. Step 5 can take place prior to the 
imaging session. Four bands are currently preferred for such 



visualization. Filter bands j=3 and 8, in a 3:2 ratio, for the 
red (R) channel, filter band j=2 for the green (G) channel, 
and filter band j=9 for the blue (B) channel, in Table 1. As 
indicated in Step 6 of FIG. 3(a), the relative weights applied 
to the R:G:B channels are preferably inversely proportional 
to <Igray strip> / the average intensity over the portion 
occupied by the gray strip area in each image. This proce- 
dure tends to reconstruct the hues and saturations in the 
original scene to within accuracy limits associated with 
response nonlinearities of the display 19. To minimize the 
effects of such nonlinearities with display monitors such as 
the Sony Model GDM-175E1 Multiscan monitor, for 
example, the viewer may prefer to adjust the maximum 
brightness in the image to correspond to the maximum 
image intensity level of the monitor. A linear transformation 
step, which can be readily accomplished by commercial 
software such as Adobe Photoshop, may be used. If the 
digital images are derived from photographic slides, as in the 
embodiment of FIG. 1(c), steps 5 and 6 are not necessary. 

As indicated by dashed lines in FIG. 3(a), either the 
normalized monochromatic images resulting from Step 4 or 
the color visualization provided from Step 6 can be dis- 
played on the display 19. Any or all of the monochromatic 
raw images could be displayed as well. 

FIG. 3(b) is a flow chart of a preferred method of 
processing images according to the present invention for 
characterizing the condition of a region of interest of the skin 
of a subject which includes a skin lesion. A skin lesion is 
selected at Step 50. Digital images of the lesions illuminated 
by light filtered at the desired wavelengths of X,-A, 4 . . . , are 
digitally recorded in Steps 52, 54, 56 and 57. . . , as described 
above. Each of these digital images is processed separately. 
In Step 58, the image taken at the lowest wavelength band 
is used to create a mask for segmentation. At Steps 60, 62, 
64, 65 ... , each of the images of the lesion that correspond 
to different wavelengths are segmented by means of the 
segmentation mask obtained at Step 58. Lesion parameters 
are computed from each of the segmented images, in Step 
66. Lesion parameters found to be useful for classifying and 
characterizing the lesion and statistical methods for com- 
puting the estimated values of the parameters, are discussed 
further, below. The estimated values of the parameters are 
provided to a linear classifier in Step 68. The linear classifier 
employs a linearly weighted sum of the individual param- 
eters to derive a value used to classify the lesion as malig- 
nant or benign. A non-linear classifier such as a Gaussian 
quadratic classifier or an artificial neural-net classifier, each 
employing a suitable defined merit function, may be used as 
well. In either case, the numerical value produced by the 
classifier is subjected to a threshold test at Step 100, such 
that if the test is passed, the lesion is suspected to be 
malignant melanoma. If the test is failed, the lesion is 
declared not to be melanoma. The lesion could also be 
characterized as invasive or non-invasive with a different 
classifier. 

I. SEGMENTATION 

The segmentation algorithms will now be described. The 
function of the segmentation algorithms is to discriminate 
between the lesion and normal skin in the field-of-view of 
the imaging device. This is a complex function since not 
only is the lesion appearance highly variable but so is the 
appearance of healthy skin due, for example, to the presence 
of blotches, hair, wrinkles, etc. The automatic algorithm 
described here is based on the images in the blue spectral 
band, from about 400 nanometers (nm) to 500 nm. This 
spectral band was selected because melanin absorption 
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increases rapidly as the wavelength decreases. While the use respectively. FIGS. 7(b) and 7(e) are images resulting from 

of ultraviolet radiation could be advantageous, since ultra- the iterative thresholding described above. Various dark 

violet radiation is carcinogenic, only low doses can be used. blobs are seen in the images outside of the lesion area. These 

Segmentation in blue consists of several automatic steps: are removed in the following step. 

Location of major peaks in the histogram 5 Image cleaning . . . . 

First, the histogram of intensity levels in the whole image Some of the blobs in the tnre sholded images arise natu- 

is determined. TTien given a sliding window with the range ^ due ^ tQ ^ Qn ^ norma , skin or tQ hair 

of (2L+1) intensity levels, the number of peaks N„ in the . ^l. r u i. ^ j 

u - * *u * • i , rc C T ~ J* as in FIG. 1(b). Others are artifacts such as the film edge at 

histogram over that range is determined. If N„<2, the range lft , . , . . m „ x , , , , & . 

is decreased by two levels and if N„>3, the range is increased 10 the t0 P of me nevus ima S e in ™ G ' 7 «' or dark bands at 

by two levels and the process is repeated until N^=2 or 3. For ima § e ed S es from tne sllde mounts. These bands are 

most of the images in the data base used in this study, there removed by automatically testing for their presence and then 

are two major peaks in the histogram. Examples of such setting the intensity of appropriate pixels to zero. The 

histograms are shown in FIG. 4(a) for a malignant mela- remaining blobs could also be removed by determining the 

noma and in FIG. 4(b) for an atypical melanocytic nevus. overall number and size, i.e., number of pixels, of connected 

The lesions correspond to the lower intensity peak, since it blobs, and then setting to zero the intensity of pixels 

is darker than the surrounding skin due to strong absorption belonging to the small ones. However, since the size of some 

by melanin at 400 nm. However, some lesions are quite lesk)ns exceeds m Q00 p ixe ls, this would be computation- 

inhomogeneous, and the automatic procedure descnbed can _ tl ™ , . , . 

find 3 major peaks, as illustrated in FIGS. 5(a) and 5(b). ~° ally very ,n,ensive - ™° re > m P ractice ' ste P 1S 

j . c , . . i l j preferably earned out as follows. First, perimeter pixels for 

Location of the intensity threshold , : , . . , ^_ , , . . , 

, all blobs m the image are located. The number of such pixels 

If two major peaks are found in the intensity histogram, is icall , ess man {0 QQ0 ^ each of ^ 

then the threshold value I„ is selected to be at ^t^ram ^ is ^ d tQ a m{ Wob and its si J number 

minimum between these two peaks, as indicated in FIGS. 25 r r . . . . .7 . . , , ^ . 

4(a) and 4(b). In the case of three peaks, it has been found of penmeter pixels in the blob, is determined ™ e f 

mat, if me middle^ is closer to the lowest intensity peak, of ^ els belonging to blobs of size less than 30% of the 

the threshold value is at the minimum between the middle m aximum size for that image are set to zero. This process is 

and the highest intensity peak. If the middle peak is closer llerated 1111X11 m the sma11 blobs m amoved. Typically less 

to the highest intensity peak, then the threshold value is at 30 than 10 derations are needed. The intensity of all the nonzero 

the minimum between the middle and the lowest intensity P ixels is tnen set to 1. The resulting binary lesion mask has 

peak, as shown in FIGS. 5(a) and 5(b). the following property: 

Iterative thresholding of the image 

' The next step in image segmentation is iterative thresh- /, ( . Y , y) = j *• * pixel al (A ' » belongs t0 lesio « (4) 

olding of the images. Given the intensity threshold value, 35 otherwlse - 
image thresholding has been typically accomplished as 

follows. The intensity I(x,y) of a pixel at location (x,y) is set , .„ , , . 

to zero if it exceeds I„, i.e., FIGS - 7c > and 7 ^ dlustrate the resulting lesion masks. 

In the images illustrated in FIGS. 1(a) and 7(d), dark hairs 

(i(x,y), if Hx, y) < i th : (i) 40 were either absent or were not adjacent to the lesion. 

4(.v, *) = | a otherwise However, there are many images with prominent dark hair 

overlapping lesions. Segmentation of such images is 

described in the following section. 
FIGS. 6(a) and 6(d) are examples of digital images of c r u - 

malignant melanoma and atypical melanocytic nevus in the 45 Segmentation of images in presence of hair 
blue spectral band, respectively. FIGS. 6(b) and 6(e) are FIGS - 8 ^ ^ 8 W m examples of lesion images with 
images resulting from the direct thresholding as in Eq. (1). nair - Since * e segmentation algorithm described in the 
As shown in FIGS. 6(b) and 6(e) y "holes" can appear within previous section would leave some of these dark hairs 
the lesion. Therefore, an iterative approach is preferably connected to the lesion, images with hair require special 
used. First, the intensity of pixels at the image edges is set 50 preprocessing to allow for hair removal from the normal 
to zero. Then as each iteration proceeds, the intensity I(x,y) skin. Since hair is a problem because of its high contrast with 
of a pixel at location (x,y) is set to zero if it exceeds \ th and respect to the normal skin in the blue, a spatial filter was 
at least one of its nearest neighbors has zero intensity, i.e., designed to locate hairs. This filter, shown in FIG. 9, is 

magnification dependent. It is applied to every pixel of the 
Il(x y ) = f 0, tf /(v ' y) * Ith md N,m = 0: (2) 55 original image and the result is tnresholded at the 5% of 

1 ' otherwise, maximum value in the whole filtered image. The filtered 

images are shown in FIGS. 8(6) and 8(f) in reverse intensity 
wnere contrast, wherein bright features are dark. Hairs are clearly 

6o located in the filtered images. It should be noted that the 
^„ w =min[/(^-i,y)j(A+i,v)j(x>^i)JUv+i),]. (3) lesion interior is almost entirely blank, indicating poor 

This procedure is iterated until there are no pixels with contrast between hair md lesion ' 

I(x,y)=I*. and a nearest neighbor with zero intensity. Hairs m removed by an averaging process. For every 

Typically, only a few iterations are required to complete this non-zero pixel at (x,y) in the filtered image one finds the 

step. The resulting images are shown in FIGS. 6(c) and 6(f). 65 locations of 4 nearest pixels (x /( y), (x M ,y), (x, y,), (x,y M ) 

FIGS. 1(a) and 1(d) are other examples of digital images (where X/<x<x u and y 7 <y<y M ) with zero intensity. Then the 

of malignant melanoma and atypical melanocytic nevus, intensity of every pixel in the original image that has 
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non-zero intensity in the filtered image is replaced as fol- 
lows: 



Uv, y) = 



(5) 



— £ [I{x u + *, y) + /(.v, - k, y) + /(.v, y„ + *) + /(.v. » - *)!■ 



The images averaged in this way are shown in FIGS. 8(c) 10 
and 8(g). It is seen that the contrast between hairs and 
normal skin is considerably reduced in these images. After 
this preprocessing, the segmentation algorithm described in 
the previous section is applied to the averaged image. The 
final binary lesion masks are shown in FIGS. 8(d) and 8(h). 15 

The preprocessing step described above may be used for 
all lesion images, regardless of the presence of hair, enabling 
fully automated lesion segmentation. However, since this 
requires more computation and causes some border blurring, 
the need for preprocessing due to the presence of dark hair 7Q 
is preferably indicated interactively by an operator, and 
images preprocessed only when necessary. 
Segmentation of images in other spectral bands 
Since melanin absorption is strongest in the shortest- 
wavelength band, the lesion area, which appears as a dark 95 
region in the image, appears largest in the blue spectral band. 
Since longer wavelength radiation penetrates deeper into 
skin, if the thickness of the melanin-containing layer com- 
pensates for the weak absorption, that part of the lesion will 
appear dark even in the red spectral band. For thick 30 
melanomas, with Breslow thickness greater than 1 mm, one 
expects dark lesions even m the infrared bands. This was 
observed, for example, by Marchesini et al, Photochemistry 
& Photobiology, "In vivo spectropho to metric evaluation of 
neoplastic and non-neoplastic skin pigmented lesions. HI. 35 
CCD camera-based reflectance imaging," Vol. 62, 1995, pp. 
151-154. However, for early malignant melanomas, with 
Breslow thickness less than 1 mm, great variability of 
images in the red spectral band has been found. There may 
be so little contrast between the lesion and the normal skin 40 
that direct segmentation is not possible. Therefore, segmen- 
tation of lesion images in all spectral bands with wavelength 
X uses the binary lesion mask of Eq. (4), obtained in the 
shortest-wavelength band, here blue, i.e., 



I L (x,y:K)==I( X ,y;\)xI B (x,y) 



(6) 



II. LESION PARAMETER ESTIMATION 

Objective and automatic lesion classification requires 
quantitative algorithms for lesion parameter estimation from 
their segmented images. Such parameters should be 
dimensionless, independent of lesion location and orienta- 
tion in the image, and of the overall image brightness. It is 
convenient to separate the parameters used here into four 
broad classes: asymmetry, blotchiness, border, and texture. 
Parameters with the highest diagnostic accuracy for malig- 
nant melanoma are listed in FIG. 11, together with the values 
of diagnostic accuracy, sensitivity, and specificity, for a 
training set of images of 41 malignant melanomas and 104 
atypical melanocytic nevi obtained with the imaging system 
described above, with respect to FIG. 1(a) wherein the 
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monochrome camera 16 was used to digitize slides. The 
subscript r, g, or b refers to the red, green, or blue spectral 
band in which the parameter is evaluated. If additional 
spectral bands are used, then each of the parameters could be 
computed at the additional spectral bands, as well. 

Specific algorithms for these parameters are described 
below. For simplicity it is assumed that the image pixels are 
square but the algorithms described below may be imple- 
mented for rectangular pixels as well. 

Lesion Asymmetry 
Asymmetry parameter 

The lesion asymmetry parameter is based on moments of 
the intensity distribution. First, the lesion orientation angle 
9 is used to locate the principal axes, which are just the 
symmetry axes for symmetric lesions. The angle 0 is com- 
puted from 



tan 20 = 



2<(.v-.v f )(y-y e )> 
«A'-A>) 2 >-<(y-,y f ) 2 >' 



where the lesion intensity centroid is at 

.\\-<\> and y c =<y>. 



(7) 



(8) 



The angular brackets in Eqs. (7) and (8) denote an intensity 
moment, which for any function f(x,y) of position in the 
image can be computed as follows: 



(fix, v.)> : 



y)t L fr y) 

x y 

IL4(*, y) ' 



(9) 



45 



where l L (x y y) is the segmented lesion image. In order to 
compare properties of different lesions, the parameters used 
are independent of the orientation of the lesion in the image. 
Therefore, the lesion asymmetry is determined with respect 
to the principal axes. The measure of asymmetry described 
here requires rotation of the image by an angle 9 so that 
principal axes are parallel to the image axes. In this 
principal-axis coordinate system the following asymmetry 
factors are defined: 



FIGS. 10(a)-10(f) are a series of images of the lesions, 
with their corresponding histograms shown in FIGS. 5(a) 
and 5(b\ segmented in the blue, green, and red spectral 
bands, as indicated. The automatically determined lesion 50 
borders are superimposed on the original lesion images. The 
area of dark regions is largest in the blue. 



A v = - 



2£|/L(Av + n, y)-4(A>-«, y)\ 
XX4tv, y) ' 

ye+n)-I L {x, y e -n)\ 
Zlhix* y) ' 



(10a) 



(10b) 



55 



The asymmetry parameter, 



A=A X +A V < 



(ID 



is a measure of asymmetry in the geometric shape of a lesion 
60 as well as in the distribution of lesion pigmentation. Asym- 
metry parameters tend to be larger for malignant melanomas 
than for atypical melanocytic nevi. 
Binary asymmetry parameter 

If the intensity distribution \ L in Eqs. (10a) and (10b) is 
65 replaced by the binary intensity distribution of Eq. (4), then 
the corresponding asymmetry parameter A bin is the fraction 
of the lesion pixels which do not have a counterpart on the 
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other side of the principal axis. Thus, when based on the 
binary intensity distribution, parameter A bin is a measure of 
the asymmetry of the geometric shape of the lesion. 

Lesion Blotchiness 

Visually, many early malignant melanomas appear 
blotchy. In multispectral images there may be darker and 
lighter regions or blotches of rather homogeneous intensity. 
In color images, in contrast, there may be regions of different 
colors. Therefore, it is of interest to quantify such blotchi- 
ness in order to differentiate malignant from benign lesions. 

Blotchiness Parameter Based on Spatial Intensity Distri- 
bution 

The lesion is divided into N, "topographic" regions. If 
I, J)fU . and \ min are the maximum and minimum intensities in 
the lesion in some spectral band, respectively, then a pixel at 
(x,y) belongs to the nth region if 



10 



15 
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channels are equal for white. In the spherical color coordi- 
nate system, 



i?(A\ y) = 
G(.v. y) = 



/rCv< y) 



(16) 



//K.v, y) + I B {x, y) + I G (x, yV 

/gU, y) 

/*(.v, v) + /j?(.v, y) + / c (.v, y) ' 



, , . , 1 max Anin _ , , 

Anin + (n - 1) - < //.U, y) < I min + n 



/ m ax - I m 



where the subscripts R, G, B refer to red, green, and blue 
spectral bands, are chosen as the independent variables. The 
lesion is then divided into color regions as follows. First 
R(x,y) and G(x,y) are divided into and N G topographic 
regions. A color region is defined as a particular combination 
of two topographic regions. The total number of color 
regions is 



Nc=N R xN G 



(17) 
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(12) The blotchiness parameter based on color is defined in 
analogy with Eq. (14): 



For nth topographic region defined in Eq. (12), a distribution 
of distances of pixels in that region from the intensity 
centroid of the binary lesion mask 



4,(a\ y) ■ 



x c ) 2 + (y n -y c ) 2 



(13) 



BI-- 



2VVatfrf„) 
»=] 

I <d„) 



(14) 



This parameter can be evaluated in every spectral band. 
Blotchiness Parameter Baled on Centroids 
The lesion is again divided into N, "topographic" regions 
as defined in Eq. (12). An intensity centroid (x c (n),y c (n)), 
defined in Eqs. (8) and (9), is then computed for each such 
region separately. The blotchiness parameter based on the 
centroid is defined as 



C-(X n , ox -X mln )(Y ma -Y min )iA, 



(15) 



25 



is obtained and its mean value <d„> and variance Var(d„) are 
computed. The measure of lesion blotchiness based on 
spatial intensity distribution is 



35 



40 
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C/ = 



^ VVar(</„) 



n-l 



(18) 
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Lesion Border 
Border Irregularity Parameter 

Border irregularity is a well-known feature of malignant 
melanomas. It is typically defined as the ratio of the mea- 
sured lesion perimeter to the perimeter of a circle with the 
same area as the lesion. Since perimeter is difficult to 
estimate reliably, a statistical descriptor of border irregular- 
ity is used here. In addition, many lesions are elongated and 
an ellipse is a better approximation for such lesions with 
regular borders than a circle. 

Using the binary lesion mask of Eq. (4), the lesion 
intensity centroid from Eq. (8), orientation angle from Eq. 
(7), area, and the aspect ratio defined as 



ar = 



50 



>l(y-yc) 2 



(19) 



where, for example, X mav is the maximum value of x c (n), 
and A; is the lesion area in pixels. This blotchiness parameter 
is also determined in each spectral band separately. 

Blotchiness Parameter Based on Spatial Color Distribu- 55 
tion 

The "color" in this analysis is not related to the visual 
perception of color. It is a quantitative descriptor of the 
relative intensities in red, blue, and green channels in a 60 
particular pixel. 

All the other lesion parameters described here involve 
analysis of images in each spectral band separately. 
Therefore, absolute calibration of image intensities was not 
necessary. However, in order to describe the color 65 
distribution, normalization of intensities in red, green, and 
blue spectral bands is needed, so that intensities in the three 



where primes refer to the coordinate system defined by the 
lesion principal axes, are determined. These values are then 
used to construct an ellipse that is the best regular approxi- 
mation to the lesion border. For each lesion border pixel at 
(x 6 ,y 6 ), its angle with respect to the horizontal axis: 

a , -i <-**-■*«■) (20) 
d> — tan , 

(yt-yc) 

and the location of the ellipse border for the same angle 
(x e (4>)y e (<t>)) are determined. The distribution of distances 
between the ellipse border and lesion border: 

UwMtv^MW- (2 1 ) 
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where 



20 

-continued 



dbixb, yb) = V \xb - a>) 2 + {yb - >v) 2 
and 

d c (4>) = V-v 2 +y- , 



(22) 



and 



^ 72 = VVar(C,) 

( * 3) Another texture parameter uses the following variable: 



(29) 



is obtained and the border irregularity parameter is defined i° 
as 



C 3 (.v,y )=4^(0,OH-w'(-;/,.OHM*(/^0>+w'(0.-;? f H>t'(0./i / ) -2 [u'(-H r -/i,H- 
u<-H r ji,)+H>(/(,.-«,)+u'(/i r «,)] (30) 



where 



<4>> 



Border Gradient Parameter 



(24) 



H'(/,y>/(.v+/'.y+/)//(.v,v). 



(31) 



Another parameter that quantitatively characterizes lesion 
border is the measure of intensity gradients across the lesion 
borders over the length scale defined by n 5 , in units of pixels. 20 
For each lesion border pixel at (x b ,y b ) one determines 
whether pixels at (x^+n^y^+n^,) are at the border. If they are 
not, then the gradient is defined as 

ax^ fc >==W[|/(.r+fl,,yW(AWi,.y)M/U^gWUy-n 4 )|]: (25a) 25 

otherwise, if pixels at (x±n £ ,y) are not on the border, 



If the value of C 3 is negative, it is set to zero and the 
corresponding texture parameter is 



T3 = 



WaKCj) 

<C 3 > 



(32) 



Another variable that leads to a texture parameter useful for 
classification of melanocytic lesions is: 

C 4 (A,v)=8>v(0.0>->v(-/i,.0)-iv("rO)-w'(0-/j / )-n'(0,;/,) -vK-H r -/i,>- 
w(-n t Ji t )-w(n r -n t )-w(n r n r ) (33 > 



G(x b ,y b t=\I(x+n g .y)-I(.x-n g ,y)\, 

or, if pixels at (x,y±n g ) are not on the border, 

G(x b .y b )=\I(x,y+n g }-I(x,y-n s )\. 
The border gradient parameter is defined as 

^ VVaitG) 
O = — ■ . 

(G) 



(25b) Again, if the value of the variable is negative it is set to zero 
30 and the corresponding texture parameter is 



(25c) 



T4- 



VVar(C4) 
<C 4 > 



(34) 



(26) 



Lesion Texture 

The description of lesion texture is particularly vulnerable 
to subjective judgement. The quantitative evaluation of 
lesion texture parameters is possible only using computer- 
based image analysis. While many such parameters are 
possible, those found to be helpful in discriminating 
between malignant melanomas and atypical melanocytic 
nevi are described below. 

Texture Parameters Based on Local Intensity Variations 

Texture parameters are defined over a length scale n, in 
units of pixels. For example, consider a pixel located at (x,y) 
in the lesion. Let I, and I M be the minimum and the maximum 
intensities in an image in the 2n,+lx2n f +l window around 
this pixel, i.e., in the range [x-n,, x+nj and [y-n„ y+nj. 
Consider a variable 



35 

Texture Parameters Based on Pigmented Network 
Texture parameters have also been developed by consid- 
ering the properties of a pigmented network. These texture 
parameters are measures of variability in the area of the 
40 dermal papillae and in the aspect ratio (length/width) of the 
rete ridges. 

Since dermal papillae appear as the brighter part of the 
network, one seeks all the local maxima over a 2n,+lx2n,+l 
window. Starting from such a maximum at (x m ,y m ), one 
finds local one-dimensional minima in eight directions (2 
vertical, 2 horizontal, and 4 diagonal) and locates the 
vertices of an octagonal region one pixel closer to the 
maximum intensity pixel than the minimum pixel. Such 
octagonal regions approximate the areas of dermal papillae 
A dp which are computed from the known location of verti- 
ces; the corresponding texture parameter is 
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The first two texture parameters are defined as: 



ti 



VVaitC!) 
(C,> 



(28) 



65 



Some of the areas determined by this algorithm are due to 
bubbles visible in some of these dermoscopic images. 
However, since there are typically on the order of hundreds 
of areas, and on the order of tens of bubbles, the statistical 
parameters should not be significantly biased by this artifact. 

The aspect ratio of rete ridges is determined in a similar 
fashion, although one starts with local minima since rete 
ridges appear dark in the images. The vertices of an octago- 
nal region are determined in mis case from one-dimensional 
maxima in the eight directions. The maximum and minimum 
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extents of this region are then determined and the aspect 
ratio R is computed. This texture parameter then is 



T6-. 



VVarf/?) 



(36) 



III. LESION CLASSIFICATION 

Selection of lesion parameters for classification was done 
by determining the maximum diagnostic accuracy for malig- 
nant melanoma for each parameter computed in every 
spectral band available for the training set of images. As 
mentioned above, diagnostic accuracy, sensitivity to malig- 
nant melanoma and specificity for the selected twenty two 
parameters are shown in FIG. 11. These parameters were 
then used as input to the linear classifier. Nonlinear classi- 
fiers may be used as well. 

For each lesion k the linear classifier is 
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where X„(k) are the parameters for the kth lesion and 
weights w„ are to be determined so that a specified function 
F(L) attains maximum value. The following functions F(L) 
were used: 1) specificity under constraint of 100% sensitiv- 
ity to malignant melanoma for the training set which 
included 41 malignant melanomas and 104 atypical melano- 
cyte nevi; (2) classification accuracy for the 24 invasive and 
16 noninvasive malignant melanomas of the training set; and 
3) correlation with the Breslow thickness for the 24 invasive 
malignant melanomas. 

Given any training set of lesion images and corresponding 
set of lesion image parameters, the weights that maximize 
F(L) are found as follows. First, an initial range and reso- 
lution A for w„ are selected. For each allowed set of values 
of w„, the values L(k) are computed for each lesion. The 
value F(L) is determined based on the input from histo- 
pathological evaluation of the lesion based on a biopsy, such 
as the diagnosis of the lesion as benign or malignant, and the 
Breslow thickness for a malignant melanoma The range of 
w,/s is adjusted until the maximum value of F(L) is inside 
the range. Then the resolution A is reduced by a factor of 
two, and the process is repeated until A reaches specified 
minimum value A m/rt . This procedure determines the weights 
w„ only up to a multiplicative constant. It is noted that the 
classifiers resulting from a particular training set are appli- 
cable only to images with a specific spatial and spectral 
resolution, and that lesion images obtained with a different 
imaging system may require the development of different 
classifiers, by the procedures described above. 

Since detection of melanoma in its early stage signifi- 
cantly improves prognosis, there is a need for reliable 
methods of early detection. Clinical evaluation of melano- 
cyte lesions is, however, a problem since reliable differen- 
tiation between early malignant melanoma with Breslow 
thickness less than 1 mm and atypical melanocytic nevus is 
difficult even for experienced dermatologists. In order to 
detect as many early melanomas as possible, weights in the 
linear classifier are preferably chosen to maximize specific- 
ity under the constraint of 100% sensitivity to malignant 
melanoma for the training set. For each set of weights, one 
finds the threshold value L rh of the linear classifier such that 
a kth lesion is classified as suspicious of malignancy if 
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L(k)>L r/l , and as benign otherwise. The resulting classifier 
for the training set is 

L i ^m5A bin ^WOA b ^mA g +OA60A l M),mC h M).095Cl+ 
0.038B-K). 1077^+0.0647^0.0 1 872 r +0. 1 1 1 73 /r M). 1 6773 s + 
J 0.26875 t (38) 

where the weights are normalized so that the threshold value 
equals one. This classifier with sensitivity to malignant 
melanoma of 100% and specificity of 85% is shown in FIG. 

10 12. Statistical significance of the specificity and sensitivity 
was assessed by considering the binomial probabilities for 
the value of L, to exceed the threshold for the 41 malignant 
melanomas and 104 atypical melanocytic nevi of the train- 
ing set separately. At the 95% confidence level, one finds 

15 that sensitivity is not less than 93% while specificity is not 
less than 79%. Since there are several melanomas very close 
to the threshold value, a practical classifier may use a 
threshold value that is less than one. It has been found that 
this set of 145 images is sufficient to yield statistically 

20 significant results. A greater number of images may be used, 
as well. 

Some of the noninvasive melanomas, called melanomas 
in-situ, are confined to epidermis and are 100% curable by 
surgery. The invasive melanomas, i.e., superficial spreading 

25 melanomas in our data base, require more extensive surgery. 
Therefore, it is of clinical interest to differentiate between 
invasive and noninvasive melanomas and a linear classifier 
was trained to perform this task. This classifier, with weights 
chosen to maximize the over-all classification accuracy for 

30 the 24 superficial spreading melanomas and 16 melanomas 
in-situ of the training set is 
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where a constant was subtracted from the classifier values to 
obtain the threshold value of zero. This classifier, with 
overall classification accuracy of 92.5%, is shown in FIG. 
13. 

40 Since prognosis for invasive melanomas correlates 
strongly with the Breslow thickness, a linear function of 
lesion parameters Q was trained to maximize the Pearson 
correlation co-efficient between Q and the Breslow thickness 
for a set 24 superficial spreading melanomas. This function 
is 
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and is shown in FIG. 14. Even though there are only 24 
50 superficial spreading melanomas in the data base, the high 
correlation of 0.77 is statistically very significant since 
p=9xl0" 6 

The classifiers of Eqs. (38H40) are applicable to the 
imaging system described above, with respect to FIG. 1(a) 
55 wherein the monochrome camera 16 was used to digitize 
slides. 

For other imaging systems, having different spatial and 
spectral resolution, different classifiers may need to be 
developed, based on a sufficient data base of lesion images 

60 obtained with that imaging system, in accordance with the 
procedures described above. 

The segmentation, parameter estimation and classification 
programs described in Sections I — III, above, can be imple- 
mented on any personal computer, using a programming 

65 language, such as FORTRAN or C. The program can be 
stored on any convenient media readable by a computer, 
such as read only memory, ("ROM"), random access 
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memory with a battery backup, electrically programmed 
ROM (EPROM), electrically erasable ROM (EEPROM), 
floppy disc, CD ROM, or a hard disc. Other suitable media 
may be used, as well. 

While the procedures of Sections I — III were described 
with respect to digital images obtained by imaging color 
photographic slides of skin lesions with a monochrome CCD 
camera 16 in accordance with the system of FIG. lc, these 
procedures are readily adaptable to the analysis of digital 
images of skin lesions acquired directly from the region of 
interest of the skin with a monochrome CCD camera 6 of 
FIG. la and FIG. 2. 

In the process described in Sections I — III, above, segmen- 
tation was conducted in the blue wavelength band. The 
segmented mask in blue was then applied to images in the 
red and green wavelength bands. Where images at additional 
wavelengths are provided, segmentation is preferably first 
attempted at the shortest available spectral band where the 
contrast between the lesion and normal skin tends to be 
highest because of strong absorption by melanin. The seg- 
mented mask is then applied to the images in the remaining 
spectral bands. These steps are shown in FIG. 3b. 

In addition, where parameters are described in terms of 
the red, green and blue wavelength bands in Section I — III, 
parameters can be derived at each of the other wavelengths 
used in the systems and methods above, in accordance with 
the procedures described in Section I— III. The additional 
parameters can be readily used to develop a classifier, also 
by the processes described in Section I — III. 

The references cited above are incorporated by reference, 
herein. 

While preferred systems and methods for practicing the 
present invention have been described above, it is under- 
stood that departures may be made from the systems and 
methods, without departing from the scope of the present 
invention, which is defined by the following claims. 

We claim: 

1. A method of characterizing a skin lesion wherein the 
absorption and scab of light in different spectral bands by the 
skin lesion is a function of the condition of the skin, the 
method comprising: 

illuminating a portion of the skin including the region of 
interest by light in at least three spectral bands; 

digitally imaging a portion of the skin including the region 
of interest at the at least three spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 
skin; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital image by generating a single seg- 
mentation mask defining the boundary of the region of 
interest for each image, where the single segmentation 
mask is the segmentation mask having largest area of 
segmentation masks generated from each image in each 
of the at least three spectral bands, without operator 
intervention; 

automatically computes at least one estimated value for 
each digital image at each spectral band which is a 
function of a characteristic of the portion of the region 
of interest determined by the segmentation mask, with- 
out operator intervention; 

characterizes the condition of the skin as malignant or 
benign based on the estimated values, without operator 
intervention; and 

outputs the characterization of the condition of the skin. 
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2. The method of claim 1, further comprising estimating 
at least one value which is a function of the texture of the 
region of interest. 

3. The method of claim 2, wherein the computing step 
5 comprises estimating values which are statistical measures 

of local intensity variation in the digital images in each 
spectral band, which are a function of the texture of the 
region of interest. 

4. The method of claim 2, wherein the computing step 
10 comprises estimating values based on the ratio of standard 

deviation of the areas of dermal papillae to their mean within 
the segmentation mask. 

5. The method of claim 2, wherein the computing step 
comprises estimating values of the average and standard 
deviation of the thickness of rete ridges within the segmen- 
tation masks. 

6. The method of claim 1, further comprising estimating 
at least one value which is a function of the asymmetry of 
the region of interest in each spectral band, for two principal 
axes of the segmented image by: 

determining the principal axes of the segmented image; 
rotating the principal axes of the segmented image until they 

are oriented parallel to the coordinate axes of the image; 
computing the differences in intensity between each pair of 
l5 pixels whose locations, with respect to a principal axis, 

are mirror images of each other; 
summing the absolute values of said intensity differences; 
calculating asymmetry values with respect to each principal 

axis, by normalizing the said sum to the total intensity in 
JQ the segmented images; and 

adding together the asymmetry values calculated for the two 

principal axes. 

7. The method of claim 1, further comprising estimating 
at least one value which is a function of the blotchiness of 
the region of interest. 

8. The method of claim 1, further comprising estimating 
at least one value which is a function of the irregularity of 
the border of the region of interest by estimating a value 
which is a statistical measure of the deviation of the border 
of the segmentation mask from the border of an ellipse of the 
same area, aspect ratio, and orientation as the segmentation 
mask. 

9. The method of claim 1, further comprising estimating 
a value which is a function of the gradient at the border of 

^ the region of interest by estimating a statistical measure of 
the gradient values of the intensity of the digital images 
across the border of the segmented images, at each spectral 
band. 

10. The method of claim 1, further comprising character- 
;Q izing the type of lesion as invasive or non-invasive. 

* 11. The method of claim 1, wherein the segmenting step 
comprises generating the segmentation mask from a digital 
image by: 

removing digital signals from the digital image which 
;5 correspond to hair structures; 

deriving a threshold from a multimodal histogram of 

intensity levels; 
iteratively applying the threshold to the digital signals of 
the digital image; and 
60 removing digital signals which correspond to small blob- 
like regions from the masked image. 
12. The method of claim 1, wherein the digital imaging 
step further comprises digitally imaging the region of inter- 
est with a digital camera. 
65 13. The method of claim 1, further comprising: 

photographing the region of interest with a color camera 
to form color photographic slides; and 
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illuminating the color photographic slides with light in 
each spectral band; 

wherein the digital imaging step comprises digitaUy imag- 
ing the illuminated color photographic slides of the 
region of interest with a digital camera. 5 

14. The method of claim 1, where the characterization 
step is based on a non-linear combination of the estimated 
values. 

15. The method of claim 1, where the characterization 
step is based on a linear combination of the estimated values. 10 

16. The method of claim 1, where the characterization 
step is based on a sequential combination of applying a 
linear combination of the estimated values and a non-linear 
combination of estimated values. 

17. A method of characterizing the condition of a region 15 
of interest of skin, wherein the absorption and scattering of 
light in different spectral bands by the region of interest is a 
function of the condition of the skin, the method comprising: 
illuminating a portion of the skin including the region of 

interest by light in at least thee spectral bands; 20 
digitally imaging the portion of the skin including the region 
of interest at the at least tree spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 25 
skin; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital images by generating a single seg- 
mentation mask defining the boundary of the region of 
interest for each image, where the single segmentation 
mask is the segmentation mask having largest area of 
segmentation masks generated from each image in each 
of the at least three spectral bands; 

computes at least one estimated value for each digital 
image at each spectral band which is a function of a 
characteristic of the region of interest determined by 
the segmentation mask; 

characterizes the condition of the region of interest of the 40 
skin based on the estimated values; and 

outputs the characterization of the condition of the region 
of interest of the skin. 

18. The method of claim 17, wherein the estimating and 
characterizing steps are conducted without the intervention 45 
of an operator. 

19. The method of claim 17, wherein the segmenting step 
is conducted without the intervention of an operator. 

20. The method of claim 17, wherein the illuminating step 
further comprises illuminating the region of interest with 50 
light in at least one spectral band which penetrates to the 
papillary dermis and is re-emitted therefrom. 

21. The method of claim 20, wherein the digital imaging 
step further comprises digitally imaging the region of inter- 
est with a digital camera. 55 

22. The method of claim 20, wherein the illuminating step 
further comprises illuminating the region of interest with 
light in a near infrared spectral band. 

23. The method of claim 17, further comprising suppress- 
ing specular reflections prior to the digital imaging step. 60 

24. The method of claim 23, wherein the processor 
converts the digital signals of each of the digital images into 
values corrected for non-uniformities of illumination and of 
response prior to the segmenting step. 

25. The method of claim 17, further comprising: 65 
photographing the region of interest with a color camera 

to form color photographic slides; and 
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illuminating the color photographic slides with light in 
each spectral band; 

wherein the digital imaging step comprises digitally imag- 
ing the illuminated color photographic slides of the 
region of interest with a digital camera. 

26. The method of claim 17, wherein the segmenting step 
further comprises applying the segmentation mask to the 
digital images in the other spectral bands. 

27. The method of claim 17, wherein the segmenting step 
comprises generating the mask at the shortest available 
wavelength. 

28. The method of claim 17, wherein the illuminating step 
comprises illuminating the region of interest by light in at 
least one spectral band whose center is between about 400 
to about 500 nanometers, and the segmenting step comprises 
generating the mask from a digital image at the spectral band 
between about 400 to about 500 nanometers. 

29. The method of claim 19, wherein the segmenting step 
comprises generating the segmentation mask from a digital 
image by: 

removing digital signals from the digital image which 
correspond to hair structures; 

deriving a threshold from a multimodal histogram of 
intensity levels; 

iteratively applying the threshold to the digital signals of 
the digital image; and 

removing digital signals which correspond to small blob- 
like regions from the masked image. 

30. The method of claim 19, wherein the segmenting step 
comprises generating the segmentation mask by comparing 
estimated values which are a function of textures within the 
digital images with a threshold. 

31. The method of claim 30, further comprising generat- 
ing the segmentation mask by comparing the estimated 
texture values to a threshold derived through statistical 
analysis of each digital image. 

32. The method of claim 17, wherein the computing step 
comprises estimating at least one value which is a function 
of the texture of the region of interest determined by the 
segmentation mask. 

33. The method of claim 32, wherein the computing step 
further comprises estimating values which are a function of 
the texture of the region of interest determined by the 
segmentation mask separately in each spectral band, based 
on the same segmentation mask. 

34. The method of claim 32, wherein the computing step 
comprises estimating values which are statistical measures 
of local intensity variation in the digital images in each 
spectral band which are a function of texture. 

35. The method of claim 17, wherein the computing step 
further comprises estimating a value which is a function of 
the asymmetry of the segmented image in each spectral 
band, for two principal axes of the segmented image. 

36. The method of claim 35, wherein the computing step 
further comprises: 

locating the principal axes by computing an orientation 
angle; 

computing the intensity centroid; 

rotating the digital image such that the principal axes are 

parallel to the image .axes; 
estimating asymmetry values for each principal axis based 

on the intensity centroid; and 
summing the estimated asymmetry values for the two 

principal axes. 

37. The method of claim 36, wherein the computing step 
further comprises computing the intensity moment with a 
binary intensity distribution. 
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38. The method of claim 17, wherein the computing step 
further comprises estimating at least one value which is a 
function of the blotchiness of the segmented digital image, 
the estimated blotchiness value being defined through sta- 
tistical properties of the spatial distribution of topographic 5 
regions of the digital images at each spectral band. 

39. The method of claim 38, wherein the computing step 
further comprises determining the centroids of topographic 
regions of the segmented digital image at each spectral band. 

40. The method of claim 17, wherein the characterizing 10 
step comprises comparing a weighted combination of esti- 
mated values against a threshold value. 

41. The method of claim 40, wherein the condition of the 
region of interest to be characterized is the presence of a 
melanoma and weight coefficients for each estimated value J5 
and the threshold value are selected to maximize specificity, 
under the constraint of a defined sensitivity to melanoma, on 

a representative set of training images. 

42. The method of claim 17, where the characterization 
step is based on a non-linear combination of the estimated ^ 
values. 

43. The method of claim 17, where the characterization 
step is based on a linear combination of the estimated values. 

44. The method of claim 17, where the characterization 
.step is based on a sequential combination of applying a ^ 5 

linear combination of the estimated values and a non-linear 
combination of estimated values. 

45. A system for characterizing the condition of a region 
of interest of skin, comprising: 

a source of illumination of light in at least three spectral 3Q 
bands; 

a camera for acquiring digital images of the region of 
interest based on the light re -emitted from the illuminated 
region of interest at each of the spectral bands, the digital 
image comprising digital signals whose values are a 35 
function of the condition of the region of interest; 

memory for storing the digital images provided by the 
camera; 

a digital processor programmed to perform the steps of: 

segmenting the digital images stored in memory by gen- 40 
erating a single segmentation mask, where the single 
segmentation mask is the segmentation mask having 
largest area of segmentation masks generated from 
each image in each of the at least three spectral bands; 

estimating at least one value for each digital image at each 45 
spectral band which is a function of the texture of the 
portion of the region of interest determined by the 
segmentation mask; 

characterizing the condition of the skin based on the 
estimated values; and 50 

outputting the characterization of the region of interest. 

46. The system of claim 45, further comprising means for 
suppressing specular reflections from the region of interest. 

47. The system of claim 45, further comprising means for 
calibrating each digital image to provide correction for 55 
non-uniformities of illumination and response. 

48. The system of claim 45, wherein the digital processor 
is coupled to the source of illumination and to the camera for 
controlling the intensity of illumination and exposure times, 
respectively. 60 

49. The system of claim 45, wherein the processor applies 
the segmentation mask derived from the digital images at 
one spectral band to the digital images at the other spectral 
bands. 

50. The system of the claim 45, where in the processor 65 
estimates values separately from, digital images at each 
spectral band based on the segmentation mask. 



51. The system of claim 49, wherein the processor com- 
pares a weighted combination of estimated values against a 
threshold value. 

52. The system of claim 45, wherein the camera records 
monochromatic images and the illumination means com- 
prises: 

a tungsten halogen light source with feedback to stabilize 

the intensity in each wavelength band; 
means for sequentially filtering the light; and 
an optical fiber ring illuminator to distribute the filtered 

light. 

53. The system of claim 45, further comprising a feedback 
loop for stabilizing the intensity of the light source by the 
processor. 

54. The system of claim 45, wherein the filter means 
comprises a plurality of interference filters mounted on a 
wheel for stepping any filter into a position intercepting the 
light from the light source. 

55. The system of claim 45, wherein at least one of the 
spectral bands has a center which is between about 400 to 
about 500 nanometers, and at least one other band centered 
elsewhere in the visible region. 

56. The system of claim 55, wherein the set of interfer- 
ence filters includes a filter whose center lies in at least one 
spectral band in the near infrared range whose center lies 
between about 750 and 1000 nanometers. 

57. The system of claim 50, wherein the camera is a 
single-chip, charge-coupled device and the control means 
comprises a digital computer including means for determin- 
ing exposure times for the camera which maximize the 
signal-to-noise ratio in the image at each spectral band. 

58. The system of claim 45, wherein: 

the source of illumination provides broad-band ("white") 
light; and 

the camera comprises multiple charge-coupled devices 
which detect light in a plurality of spectral bands 
between the near ultraviolet to near infrared. 

59. The system of claim 45, wherein the processor esti- 
mates values which are statistical measures of local intensity 
variation in the digital images in each spectral band, which 
are a function of die texture of the region of interest. 

60. The system of claim 45, wherein the processor esti- 
mates values based on the ratio of standard deviation of the 
areas of dermal papillae to their mean within the segmen- 
tation mask. 

61. The system of claim 45, wherein the processor esti- 
mates values of the average and standard deviation of the 
thickness of rete ridges within the segmentation masks. 

62. The system of claim 45, wherein the processor esti- 
mates at least one value which is a function of the asym- 
metry of the region of interest in each spectral band, for two 
principal axes of the segmented image by: 

locating the principal axes by computing an orientation 
angle; 

computing the intensity centroid; 

rotating the digital image such that the principal axes are 

parallel to the image axes; and 
estimating asymmetry values for each principal axis based 

on the intensity centroid; and 
summing the estimated asymmetry values for the two 

principal axes. 

63. The system of claim 45, wherein the processor further 
estimates at least one value which is a function of the 
blotchiness of the region of interest. 

64. The system of claim 45, wherein the processor further 
estimates at least one value which is a function of the 
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irregularity of the border of the region of interest by esti- 
mating a value which is a statistical measure of the deviation 
of the border of the segmentation mask from the border of 
an ellipse of the same area, aspect ratio, and orientation as 
the segmentation mask. 5 

65. The system of claim 45, wherein the processor further 
estimates a value which is a function of the gradient at the 
border of the region of interest by estimating a statistical 
measure of the gradient values of the intensity of the digital 
images across the border of the segmented images, at each |Q 
spectral band. 

66. The system of claim 45, wherein the processor char- 
acterizes the type of lesion as invasive or non-invasive. 

67. The system of claim 45, wherein the processor gen- 
erates the segmentation mask from a digital image by: 

removing digital signals from the digital image which 15 
correspond to hair structures; 

deriving a threshold from a multimodal histogram of 
intensity levels; 

iteratively applying the threshold to the digital signals of 7Q 
the digital image; and 

removing digital signals which correspond to small blob- 
like regions from the masked image. 

68. A system for characterizing the condition of a region 
of interest of skin, comprising: 

a source of illumination of light in at least three spectral 
bands; 

a camera for acquiring digital images of the region of 
interest based on the light re-emitted from the illuminated 
region of interest at each of the at least three spectral 3Q 
bands, the digital image comprising digital signals whose 
values are a function of the condition of the region of 
interest; 

a memory for storing the digital images; 

a digital processor including: 35 
digital processing means for segmenting the digital 
images stored in memory and computing estimated 
values of parameters which are a function of the 
segmented images, wherein the digital images are seg- 
mented by generating a single segmentation mask, 40 
where the single segmentation mask is the . segmenta- 
tion mask having largest area of segmentation masks 
generated from each image in each of the at least three 
spectral bands; 

digital processing means for automatically characterizing 45 
the condition of the tissue based on the estimated 
values; and 

means for outputting the characterization of the region of 
interest. 

69. A method of characterizing the condition of a region 50 
of interest of skin, wherein the absorption and scattering of 
light in different spectral bands by the region of interest is a 
function of the condition of the skin, the method comprising: 
illuminating a portion of the skin including the region of 

interest by light in at least three spectral bands; 55 
digitally imaging the portion of the skin including the region 
of interest at the at least three spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 60 
skin; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital images by generating a segmentation 
mask defining the boundary of the region of interest 65 
from a digital image in any one of the at least three 
spectral bands; 
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computes at least one estimated value which is a statistical 
measure of the deviation of the boundary of die region 
of interest from the boundary of an ellipse of the same 
area, aspect ratio, and orientation as the segmentation 
mask; 

' characterizes the condition of the region of interest of the 

skin based on the estimated values; and 
outputs the characterization of the condition of the region 
of interest of the skin. 

70. A method of characterizing the condition of a region 
of interest of skin, wherein the absorption and scattering of 
light in different spectral bands by the region of interest is a 
function of the condition of the skin, the method comprising: 
illuminating a portion of the skin including the region of 

interest by light in at least three spectral bands; 
digitally imaging the portion of the skin including the region 
of interest at the at least three spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 
skin; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital images by generating a segmentation 
mask defining the boundary of the region of interest 
from a digital image in any one of the at least three 
spectral bands; 

computes at least one estimated value of a statistical 
measure of the gradient values of the intensity of the 
digital images across the border of the segmented 
images; 

characterizes the condition of the region of interest of the 
skin based on the estimated values; and * 

outputs the characterization of the condition of the region 
of interest of the skin. 

71. A method of characterizing the condition of a region 
of interest of skin, wherein the absorption and scattering of 
light in different spectral bands by the region of interest is a 
function of the condition of the skin, the method comprising: 
illuminating a portion of the skin including the region of 

interest by light in at least three spectral bands; 
digitally imaging the portion of the skin including the region 
of interest at the at least three spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 
skin; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital images by generating a segmentation 
mask defining the boundary of the region of interest 
from a digital image in any one of the at least three 
spectral bands; 

computes at least one estimated value based on the ratio 
of standard deviation of the areas of dermal papillae to 
their mean within the segmentation mask; 

characterizes the condition of the region of interest of the 
skin based on the estimated values; and 

outputs the characterization of the condition of the region 
of interest of the skin. 

72. A method of characterizing the condition of a region 
of interest of skin, wherein the absorption and scattering of 
light in different spectral bands by the region of interest is a 
function of the condition of the skin, the method comprising: 
illuminating a portion of the skin including the region of 

interest by light in at least three spectral bands; 
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digitally imaging the portion of the skin including the region 
of interest at the at least three spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 5 
skin; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital images by generating a segmentation 
mask defining the boundary of the region of interest 10 
from a digital image in any one of the at least three 
spectral bands; 
computes at least one estimated value of the average and 
standard deviation of the thickness of rete ridges within 
the segmentation mask for a digital image of the region 15 
of interest determined by the segmentation mask; 
characterizes the condition of the region of interest of the 

skin based on the estimated values; and 
outputs the characterization of the condition of the region 20 
of interest of the skin for a digital image of the region 
of interest determined by the segmentation mask. 
73. A method of characterizing the condition of a region 
of interest of skin, wherein the absorption and scattering of 
light in different spectral bands by the region of interest is a ?5 
function of the condition of the skin, the method comprising; 
illuminating a portion of the skin including the region of 
interest by light in at least three spectral bands; 
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digitally imaging the portion of the skin including the region 
of interest at the at least three spectral bands with the light 
re-emitted by the portion of the skin to generate digital 
images comprising digital signals whose values are a 
function of the condition of the region of interest of the 
skin; 

calibrating each pixel location in the digital image in each 
spectral band with respect to stored images of a white 
target material having known diffuse reflectance, each of 
the stored images being an average of a plurality of 
images acquired at each spectral band, while the material 
undergoes continual in-plane motion; and 

providing the digital images to a processor, wherein the 
processor: 

segments the digital images by generating a segmentation 
mask defining the boundary of the region of interest 
from a digital image in any one of the at least three 
spectral bands; 

computes at least one estimated value for each digital 
image at each spectral band which is a function of a 
characteristic of the region of interest determined by 
the segmentation mask; 

characterizes the condition of the region of interest of the 
skin based on the estimated values; and 

outputs the characterization of the condition of the region 
of interest of the skin. 

***** 



